Quantum liquid crystals in imbalanced Fermi gas: fluctuations and fractional vortices 

in Larkin-Ovchinnikov states 
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We develop a low-energy model of a unidirectional Larkin-Ovchinnikov (LO) state. Because the 
underlying rotational and translational symmetries are broken spontaneously, this gapless superfluid 
is a smectic liquid crystal, that exhibits fluctuations that are qualitatively stronger than in a conven- 
tional superfluid, thus requiring a fully nonlinear description of its Goldstone modes. Consequently, 
at nonzero temperature the LO superfluid is an algebraic phase even in 3d. It exhibits half-integer 
vortex-dislocation defects, whose unbinding leads to transitions to a superfluid nematic and other 
phases. In 2d at nonzero temperature, the LO state is always unstable to a nematic superfluid. We 
expect this superfluid liquid-crystal phenomenology to be realizable in imbalanced resonant Fermi 
gases trapped isotropically. 



The tunability of interactions through Feshbach res- 
onances has led to a realization of an s-wave paired 
superfluidity and BCS-BEC crossover [H 0, 0], as well 
as promises of more exotic states such as gapless p- 
wavejij and periodic Fulde-Ferrell-Larkin-Ovchinnikov 
(FFLO)d, Q superfluidity in strongly correlated de- 
generate alkali gases. The latter enigmatic state has 
been thoroughly explored within a BCS mean-field 
studies@, H, and is expected to be realizable in 
a population-imbalanced (polarized) Feshbach resonant 
Fermi gas [3, 11 1. While recent experiments [l2|, Gil, have 
confirmed much of the predicted phenomenology of phase 
separation 11 , 3, HI in such systems, the FFLO states 
have so far eluded definitive observation. 

The simplest mean-field treatments 0, @, [ll[ find that 
the FFLO type states are quite fragile, confined to a nar- 
row range of polarization on the BCS side. However, mo- 
tivated by earlier studies 0, 0] and based on the finding 
of a negative domain-wall energy in an otherwise uni- 
form sing let BCS superfluid[g|, [ltfl , it has recently been 
argued that a more general periodic superfluid state that 
includes a larger set of collinear wavevectors may be sig- 
nificantly more stable. Much like a type-II supercon- 
ductor undergoes a continuous transition into a vortex 
state at a lower-critical field H c \, which is significantly 
below the thermodynamic field, here too, a Zeeman-field 
driven domain-wall nucleation (with the density increas- 
ing above the lower critical h c \ field) allows a continuous 
mechanism for a transition from a singlet paired super- 
fluid to a LO-like periodic state @, H, H Ell • 

In this scenario the SF-LO transition is of a 
commensurate-incommensurate type as can be explic- 
itly shown in one dimension (ld)0, [l3|- The im- 
posed species imbalance (excess of the majority fermionic 
atoms) can be continuously accommodated by the sub- 
gap states localized on the self-consistently induced 
domain-walls, with this picture resembling the doping 
of polyacetylene[18]. Such LO state can also be thought 
of as a periodically ordered mzcro-phase separation be- 



tween the normal and BCS states, that thus naturally re- 
places the macro-phase separation ubiquitously found in 
the BEC-BCS detuning-polarization phase diagram 11 
(Fig©. 
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FIG. 1: Polarization AN/N vs. l/(fc.pa) schematic phase 
diagram, showing LO liquid crystal phases replacing phase- 
separated (PS) regime [111]. Possible 3d transition scenar- 
ios as a function of temperature to the normal-nematic 
(N-Nm), normal- isotropic (N-I), normal-smectic (N-Sm2Q), 
and "charge" -4 superfluid-nematic (SF-i-Nm) phases are il- 
lustrated in the lower panel. 

With this motivation in mind, here we report on our 
study that is complementary to these microscopic mean- 
field investigations. Namely, assuming that the LO state 
is indeed energetically favorable over a region of a phase 
diagram, we explore its stability to low-energy fluctua- 
tions and the resulting phenomenology. 

We demonstrate that the low-energy model of the LO 
state is that of two coupled smectics, whose moduli we 
derive from the BCS theory. Thus we show that a reso- 
nant imbalanced Fermi gas, confined to an isotropic trap 
is a natural realization of a quantum (superfluid) liquid 



2 



crystal, that unlike the solid state analogs ljj 2(J 21, IH 
is not plagued by the underlying lattice potential that 
explicitly breaks continuous spatial symmetries. 

We find that while it is stable to quantum fluctua- 
tions, in 3d a long-ranged LO order is marginally unsta- 
ble at any nonzero T. The resulting superfluid state is an 
algebraic phase, characterized by universal guasi-Bragg 
peaks and correlations that admit an asymptotically ex- 
act description [23j|. In contrast, crystalline LO phases 
with multiple noncolinear ordering wavevectors are sta- 
ble against thermal fluctuations. 

As with treatments of the LO state where long range 
order is assumed 21 , 24], 25 1 , we also find an unusual topo- 
logical excitation - a half vortex bound to a half dislo- 
cation - in addition to integer vortices and dislocations, 
in the algebraic LO phase. Because our conclusions are 
based on general symmetry principles, supported by de- 
tailed calculations, they are generic and robust to varia- 
tions in microscopic details. 

In 2d and nonzero T, the state is also an algebraic 
phase, exhibiting universal power-law phonon correla- 
tions, controlled by a nontrivial exactly calculable [26j| 
fixed point. It displays short-range positional order with 
Lorentzian structure function peaks, and is thus unsta- 
ble to proliferation of dislocations. The resulting state 
is either a "charge" -4 (four-fermion) superfluid or a non- 
superfluid nematic, depending on the relative energet- 
ics of aforementioned integer and half-integer vortex- 
dislocation defects. The latter normal nematic state is 
a (complementarily described deformed Fermi sur- 
face statejUll^. 

MODEL: We now present the highlights of our cal- 
culations. We begin with a Ginzburg-Landau theory 
that captures the system's tendency to order into a finite 
wavevector paired state, with a preferred magnitude Qo 
and a spontaneously chosen direction. The correspond- 
ing free-energy density 

H = J [|V 2 A| 2 - 2Q 2 |VA| 2 ]+r|A| 2 + |-|A| 4 +^ j 2 +. . . . 



can be derived from a microscopic BCS model[6l llll. |23| 
near the upper-critical chemical potential difference, h C 2, 
with 
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that can be more generally taken as phenomenological 
parameters to be determined experimentally, n, cp, 
vp, Abcs, and m are the atomic density, Fermi en- 
ergy and velocity, BCS (h — 0) gap and atomic mass, 
respectively. Near the lower-critical Zeeman field, h c i, 
Qo(h) is expected to vanish with the species imbalance, 
as the system continuously transitions into a uniform 



singlet superfluid @, H, @|, with this and other mod- 
uli's dependences derivable via fluctuating domain-walls 
methods (29j. Above, j is the supercurrent and the last 
term is crucial for getting a nonzero transverse (to Q) 
superfluid stiffness in the LO state. From the first two 
terms it is clear that the dominant instability and fluctua- 
tions are near a finite wavevector of magnitude Qq. Thus, 
for h < h C 2, r < JQ 4 = 0.61n/e_F and the system devel- 
ops a pairing order parameter A(x) = J2q„ AQ ii e lQ "' x . 

As with other crystallization problems, the choice of 
the set of Q„'s is determined by the details of interac- 
tions and will not be addressed here. Motivated by LO 
findings 0, we focus on the unidirectional order charac- 
terized by a collinear set of Q„'s. These fall into two, LO 
and FF universality classes. The LO (FF) states are char- 
acterized by breaking (preserving) translational and pre- 
serving (breaking) time-reversal symmetries. Low-energy 
properties of such states can be well captured with a sin- 
gle ±Q pair (LO) and a single Q wavevector (FF) ap- 
proximations. 

Because it is the periodic LO state that is expected 
to be most stable @, 0, H, @, EH, we focus on this more 
interesting case and only comment in passing on the ho- 
mogeneous FF state. Within the LO approximation the 
pairing function is given by Aio(x) = A + (x)e i( ^' x + 
A_(x)e~ iQ ' x , where A± = Aqe ie±lyyi ^ are the leading 
complex order parameters, whose amplitudes deep in the 
ordered LO state can be taken to be equal and constant, 
Aq « cA BCS ln(h C 2/h), thereby focusing on the two 
Goldstone modes 9±(x). A slightly rearranged form of 
the LO order parameter Aio( x ) clarifies its physical in- 
terpretation 



A LO (x) = 2A Q e^ x >cos(Q.x + 5 . m (x)), 



(3) 



showing that it is a product of a superfluid order param- 
eter and a unidirectional, spontaneously oriented (along 
Q) Cooper-pair density wave, i.e., simultaneously ex- 
hibiting the ODLRO and smectic order. The low-energy 
properties are respectively characterized by two Gold- 
stone modes, the superconducting phase 9 SC = i(6> + +#_) 
and the phonon displacement u — sm /Q = ^(9 + — 
9J)/Q. In contrast, the uniform FF state is characterized 
by a single Aq amplitude and a Goldstone mode 9q. 

Substituting Alo(x) into Ti. we obtain a Hamiltonian 
density for the bosonic Goldstone modes of a generic LO 
state: 



Ti-LO — 
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(4) 



« § (v 2 ^) 2 + f (a M + i(Vu) 2 j +|(v l M 2 , 

where we dropped constant and fast oscillating parts, 
chose Q = Qqz, and defined phonon fields u± — ±9±/Qq 
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and the bend [K = UQlA 2 Q ~ 2AnA 2 Q / (e F Ql)) and 
compressional (B = 16JQqAq w 9.8nAg/e_F) elastic 
moduli. 

This form (valid beyond above weak-coupling micro- 
scopic derivation) is familiar from studies of conventional 
smectic liquid crvstals|30j. with rotational invariance en- 
coded in two ways. Firstly, for a vanishing 7 the gradient 
elasticity in u± (and u) only appears along Q, namely 
d = Q • V (compression) , with elasticity transverse to Q 
of a "softer" Laplacian (curvature) type. Secondly, the 
elastic energy is an expansion in a rotationally-invariant 
strain tensor combination Uqq = du± + ^(Vm±) 2 , whose 
nonlinearities in u± ensure that it is fully rotationally in- 
variant even for large reorientations Qoz — * Q of the LO 
ground state. 

A nonzero 7 = i^QoAq/to 2 ~ ^ n ^Q/( e F^BCs) 
coupling (minimized by a vanishing supercurrent V# + + 
V#_) removes the two independent rotational symme- 
tries, orientationally locking the two smectics. This 
leads to the superconducting phase combination, 9 SC = 
\{0+ + to be of a conventional XY (as opposed to 
"soft" smectic) gradient type. It is characterized by par- 
allel (ps~" ~ B/Qq) and transverse (p* =J ~ = Aj/Qq) su- 
perfluid stiffnesses, appearing in the second form of HlOi 
equivalent to the first form at low energies of interest to 
us. 

Physically, pi and pjr are superfluid stiffnesses for the 
supercurrent j = (j + + j_)/2 produced by the imbalance 
in the left (j_) and right (j_) supercurrent magnitudes 
and directions, respectively. We thus find that the LO 
state is a highly anisotropic superfluid, with 



pj/ P l = -(A Q /A BCS ) 2 



\n(h c2 /h) <C 1, (5) 



a ratio that vanishes for h — > h~ 2 . We find the FF state 
to be even more exotic, characterized by an identically 
vanishing transverse superfluid stiffness, a reflection of 
the rotational invariance of the spontaneous current to 
an energy-equivalent ground state. 

FLUCTUATIONS: The thermodynamics can be ob- 
tained in a standard way through a coherent path- 
integral. Although there are nontrivial issues of the inter- 
play between the fermionic quasi-particles and the Gold- 
stone modes, we can unambiguously show that at T = 
the superfluid and smectic orders (and thus the LO state) 
are stable to quantum fluctuations in d > l|29j|. 

In contrast, for T > 0, 9 SC (0 sm ) fluctuations diverge 
and ODLRO (smectic order) is destroyed for d < 2 
(d < 3). Consequently, we find that the LO state is 
unstable to thermal fluctuations, displaying quasi-Bragg 
(Lorentzian) peaks in 3d (2d) in its structure function. 
Thus in both cases the LO order parameter, ((3]) vanishes 
and the state is qualitatively distinct from its mean-field 
form, at low T characterized by a "charge" -4 superfluid 



Furthermore, in the presence of these diver- 
gent thermal fluctuations phonon nonlinearities in 
Ti-LOi Eq.Q become important. They qualitatively 
modify correlations on scales longer than £jvl ~ 
[#3/2/( B i/2 T )]i/(3-«J) „ fc f ; 1 [A|e J ,/(A| cs T)] 1 /(3-rf) 

(on shorter scales the harmonic description 
above applies), giving universal power-laws, e.g., 



(u(z,x)u(0,0)) 1/2 



Max [x a , z' 3 ] , controlled by a 



nontrivial low T (order 3 — d) fixed point [23j, that has 
an exact description in 2d with a = 1/2,(3 = l/3(26j]. 
In 3d, ~ e cI<3/2 /( b1/2t ) and phonon correlations 

grow as a universal power of a logarithm, a result that 
is asymptotically exact. These elastic results of course 
only hold as long as dislocations remain bound or on 
scales shorter than the dislocation unbinding scale. 

DEFECTS: We now turn to the discussion of topo- 
logical defects and corresponding phases accessible by 
their unbinding. With two compact Goldstone modes 
9 sc ,u (equivalently, 6± = ±2nu±/a), defects are labeled 
by vortex and dislocation charges (2im v , and)- Ordinary 
vortex, (27r, 0) and dislocation (0, a) are clearly allowed, 
and in terms of the two smectic displacements these re- 
spectively correspond to the opposite and same signs of 
integer dislocations in u±. When proliferated they de- 
stroy the superfluid phase coherence and smectic periodic 
order, respectively, and either one is clearly sufficient to 
suppress the conventional LO order, A^o, ©■ 

However, because a sign change in Alo due to a a/2- 
dislocation in u can be compensated by a 7r-vortex in 
9 SC (thereby preserving a single- valuedness of A LO ) 1/2- 
charge defects in 9 SC and u are also allowed, but are 
confined into (±7T, ±a/2) pairs 21, 24,[25j]. In terms of 



order parameter A 



(4) 



A 2 



1 A 2 i28 3 
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the two coupled smectic displacement fields, it+, u_ these 
correspond to an integer dislocation in one and no dislo- 
cation in the other. 

TRANSITIONS: There are thus many paths of con- 
tinuous transitions out of the LO (SF 2 -Smg) state. One 
is through an unbinding of ordinary integer (0, a) dislo- 
cations in u. This melts the smectic order in favor of a 
nematic, but retains a superfluid order, thereby trans- 
forming the LO state to a nematic "charge" -4 super- 
fluid (SF4-N111). Another path, is by unbinding integer 
(27r, 0) vortices in 9 SC . This destroys the superfluid order 
and converts the smectic positional order Q to 2Q (N- 
Sm2Q). Finally, a third route out of the LO superfluid 
is through a direct proliferation of (w, ±a/2) fractional 
vortex-dislocation pairs, that destroy both smectic and 
superfluid orders, inducing a transition to a normal (non- 
superfluid) nematic (N-Nm). For 3d these possibilities, 
determined by the relative energetics of these different 
types of defects are illustrated in Fig[TJ In 2d, the dislo- 
cation energy is finite and LO state is necessarily desta- 
bilized by thermal fluctuations to a "charge" -4 superfluid 
nematic, SF4-Nm. Upon rotation the resulting nematic 
superfluid will display ir vortices (S v • dl = h/4m), that 
(because of the nematic order) we expect to form a uniax- 
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ially distorted lattice. We note that this rich fluctuations- 
driven phase behavior contrasts sharply with a direct 
LO-N transition (described by U(l) x U(l) Landau the- 
ory H mft = r(|A + | 2 + |A_| 2 ) + A^A^ 4 + |A_| 4 ) + 
A2|A + | 2 |A_ | 2 ) found in mean-field theory. 

FERMIONS: We now turn to a discussion of the 
fcrmionic sector that we have so far ignored. Near h c2 a 
single harmonic (Q for FF and ±Q for LO states) approx- 
imation is sufficient. Unlike the simpler FF case (that can 
be diagonalized exactly with a two-component Nambu 
spinor[ll]), the LO state involves a three-component 
spinor # k = (c_k+Qf, c£j_, C-k-Qf)- Neglecting sparse 
off-resonant coupling between k and k + 2Q Cooper- 
pairs and noting that only two of the three components 
in are resonant at any one k, the approximate spec- 
trum is given by £ k>CT ,±Q = (4 + Aq) 1 / 2 - a(h ± 
with a = ±1, e fe = - n + [I = |(mt + 
and h = \{^] — Mj.)- The regions of k where -Ek,<r,±Q 
is negative corresponds to a Fermi sea (rather than the 
usual vacuum) of Bogoluibov quasi-particles in the BCS 
"vacuum" , and therefore leads to a Fermi surface of gap- 
less fermionic excitations. These are nothing but the un- 
paired fraction of the majority atoms. From the spec- 
trum above it is clear that two distinct LO states are 
possible. One, LOl exhibits a single (majority) species, 
fully polarized Fermi surface pockets. The other, L02 is 
characterized by both majority and minority fermion fla- 
vor Fermi surface pockets. The Fermi surface volume dif- 
ference is proportional to the species imbalance and the 
anisotropy encodes the long-range orientational order of 
the LO state. Because the polarization is conserved and 
Goldstone mode fluctuations are finite at zero tempera- 
ture, we expect the L01-L02 quantum transition to be 
of a simple band filling type. 

In the complementary regime near h c i, the excess 
majority atoms occupy additional states localized on 
domain- walls 0, H, H, Gil]. Because the atoms can freely 
move along and tunnel between adjacent domain-walls, 
near /i cl they exhibit a "metallic" , but highly anisotropic 
dispersion. The resulting fcrmionic spectrum is consis- 
tent with that near h C 2, £k,<r,±Q- 

The interactions between the Goldstone modes and un- 
paired majority fermionic atom, tp must now be included 
and are given by 

U js ,j ~ iVO • V> f VV> + /i.e., Hj„ n ~ (V0) 2 V>V, (6) 
Ha-p ~ [d z u+ -(Vu) 2 ]^i/) + iVu ■ ipWip + h.c. 

As with other analogous problems^, we expect these 
to lead to Landau-like damping of the Goldstone modes 
sc ,u, and a finite fermionic quasi-particle lifetime. We 
leave the study of these and other affects on the proper- 
ties of the LO states to the future. 

TRAP EFFECTS: Since near h c2 the LO period A Q = 
2ir/Qo @ is bounded by the coherence length (that near 



unitarity can be as short as ~ R/N 1 / 3 , where R is the 
trapped condensate radius and N is the total number 
of atoms), and thus <C R, in this regime the trap can 
be treated via a local density approximation (LDA) . For 
Xq <C R, LDA predicts weak pinning of the LO smec- 
tic, that can be estimated via finite size scaling, with 
trap size R cutting off (u 2 ) ~ 77 log(i?/Aq), leading to 
(Alo) ~ (<^q/R) v ^ 1 that no longer truly vanishes, 
but is still strongly suppressed. We thus expect the pre- 
dicted strong fluctuations effects to be experimentally ac- 
cessible. We note, for example, that Kosterlitz-Thouless 
phase fluctuation physics has been reported in 2d trapped 
superfluids [3l| . despite the finite trap size. However, a 
more detailed analysis of the trap effects is necessary, 
particularly near h c \ , for a quantitative comparison with 
experiments. 

To summarize, we studied fluctuation phenomena in 
a LO state, expected to be realizable in imbalanced res- 
onant Fermi gases. The LO state is a superfluid smec- 
tic liquid crystal, whose elastic moduli and superfluid 
stiffness we derived near h C 2. It is extremely sensitive 
to thermal fluctuations that destroy its long-range po- 
sitional order even in 3d, replacing it by an algebraic 
phase, that exhibits vortex fractionalization, where the 
basic superfluid vortex is half the strength of a vortex 
in a regular paired condensate. This should be observ- 
able via a doubling of a vortex density in a rotated state. 
Also under rotation, the high superfluid anisotropy ([5|) 
leads to an imbalance-tunable strongly anisotropic vor- 
tex core and a lattice highly stretched along Q. Bragg 
peaks in the time-of-flight images can distinguish the pe- 
riodic SF2-SniQ (superfluid smectic) state from the ho- 
mogeneous SF4-Nm (superfluid nematic), which are in 
turn distinguished from the N-Sni2Q and N-Nm (normal 
smectic and nematic) by their superfluid properties, pe- 
riodicity, collective modes, quantized vortices, and con- 
densate peaks. Thermodynamic signatures will identify 
corresponding phase transitions. 
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